1. Field of the Invention
This invention relates to systems and methods for 3D telepresence and other 3D display applications for moving images and more particularly to systems and methods for presenting auto stereoscopic integral holograms on a stationary direct-view display at video or sub-video rates.
2. Description of the Related Art
Video Conferencing
Video conferencing is a set of interactive telecommunication technologies that allow two or more locations to interact via two-way video and audio transmissions simultaneously. Standard video conferencing does not try to produce the illusion that connected locations are merged together for better interaction.
Telepresence
“Telepresence” refers to a set of technologies that allow a person to feel as if they were present, to give the appearance that they were present, or to have an effect, at a location other than their true location. Several techniques are used and emphasis is placed on image quality.
Telepresence may or may not include natural size imaging of the people, three dimensional reproduction of the scene, mutual interaction of the remote location or full video rate reproduction. Telepresence is a qualitative improvement over the standard video conferencing system. Telepresence systems include the elements presented in the block diagram of FIG. 1. For simplicity only a one-way channel is described. Telepresence system may have two-way channels of the same or different technologies (example: telepresence in one way and standard video conferencing the other way).
Telepresence systems include the following elements for each of the channels. An image capture system 1 takes pictures of a dynamic scene at the remote location at a refresh rate equal or superior to the image display system refreshing rate (full video or sub-video rates). An image processing unit 2 processes and transfers the captured images over a network link 3 to another location. An image data receiver unit 4 receives data sent by the image processing unit over the network link. The receiver unit can also be used for some or all the data processing. An image display unit 5 displays the images taken at the remote location to the audience at the video or sub-video rates. Full video rates of 30 Hz (30 Images per second) are desired. However, many applications may benefit from sub-video rates several Hz or even update rates of a couple seconds per hologram.
Telepresence has many applications like videoconferencing, connecting communities, hazardous environments, remote surgery and telemedicine, education, telepresence art and museum. Telepresence is of great benefit to reduce displacement cost of staff and employee, and is ecological by cutting the carbon emission and the fossil fuel consumption that would have been consumed by traveling to and from the remote location.
2D Telepresence
Musion Systems Ltd touts its Musion Eyeliner® system as “On-Stage Holographic TelePresence” that provides 3D holography. But holography is defined as the recording of a light interference pattern and the further diffraction of the light by the recorded structure. There are two aspects in holography: the recording and the replay. Neither is used by the Musion system. In fact the technology behind this system is called “paper ghost effect” and refers to the projection of the 2D image to a semitransparent screen. The image appears to the viewer as floating in thin air because the screen is not perceptible. As shown in FIG. 2, the Musion “On-Stage Holographic TelePresence” system is using a projector 10 that displays an image onto a semitransparent screen 20 so the audience 30 can see both the projected image and the background 40, giving the illusion that the image is floating in thin air.
Holography is known as the ultimate technique to represent an object or a scene in three dimensions. Holography reproduces perspective, parallax and occlusion. Actually, holography is able to reproduce the exact same light pattern as the one scattered by the real object. In the case of the Musion Eyeliner® system, the image is not three-dimensional, it is two-dimensional only. The Musion Eyeliner® system cannot reproduce perspective or parallax which are part of the realism for a telepresence display. Nevertheless, Musion telepresence system is using all the blocks presented in FIG. 1. The image capture system is a single camera, the image is processed to accommodate the orientation of the screen, the image is sent through a network link to a projector that illuminates the semitransparent screen altogether forming the image display system.
3D Telepresence
There exist telepresence systems that can reproduce the third dimension. They can be classified in two ways: stereoscopic systems and integral imaging systems.
Stereoscopic Systems
Stereoscopic systems only display two images to reproduce relief. One is intended to be seen by the left eye of the viewer and the other by the right eye. This reproduces parallax but not different perspectives; when the viewer moves the image remains static. The viewer cannot explore different perspectives of the scene, which accounts for some lack of realism in the image.
Moreover, to ensure the intended image reaches the intended eye, the audience has to wear glasses. There exist different systems where the images are coded into different colors and the glasses are filters (anaglyph), where the images are orthogonally polarized and the glasses are polarizers or where the image are temporally multiplexed and the glasses are synchronized shutters.
Integral Imaging
Integral imaging is an auto stereoscopic 3D display technique, meaning that it displays a 3D image without the use of special glasses on the part of the viewer. Integral imaging reproduces a light field that creates stereo images exhibiting parallax, occlusion and more than two perspectives when the viewer moves. There exist different techniques to produce integral images, one involves placing an array of microlenses in front of the camera and having the same kind of microlenses in front of the display. It is also possible to compute the integral image out of several images of the same scene taken at different positions or angles then replaying it with the adequate system. As shown in FIG. 3, integral imaging uses a lens array 100 in front of a display 110. Each lens 120 of the lens array covers a finite surface 130 of the display that contains pixels 140.
The limitation of the integral imaging technique for telepresence is the degradation of the resolution of the display. Indeed, each microlens in front of the display has a spatial extension that defines the horizontal and vertical resolution of the reproduced image. The resolution of the third dimension is defined by the number of pixels that are covered by the lens. If the microlens size increases, covering more pixels, the third dimension has a better resolution but the horizontal and vertical resolution are degraded (and conversely). To obtain an acceptable resolution in the three dimensions the microlenses should be extra small (equal or lower than eye resolution) but should also cover a large amount of pixels, altogether, this means those pixels should be extremely small and numerous which is not technically possible.
Integral Holography
Holography is the recording of a light interference pattern AND the diffraction of light by the recorded pattern. Holography is a technique that allows the light scattered from an object to be recorded and later reconstructed so that it appears as if the object is in the same position relative to the recording medium as it was when recorded. Alternately, holograms can be computer generated by calculating the modulation pattern that would have been formed if two beams with certain characteristics (wave front, intensity) would have crossed each other. The calculated pattern is next transferred to a medium to make the actual hologram. The image changes as the position and orientation of the viewing system change in exactly the same way is if the object were still present, thus making the recorded image (hologram) appear three dimensional. The technique of holography can also be used to optically store, retrieve, and process information.
In the optical recording setup, one beam is called the reference beam and does not carry any information and the other beam is called the object beam and is modulated in intensity and/or phase from the scene (either directly or by computer generation). The holographic recording material should be sensitive to the intensity modulation generated by the interference and have sufficient resolution to resolve the spatial frequencies of the scene. A hologram is read by sending a reading beam to the hologram and by looking at the diffracted beam. The reading geometry and the reading source characteristics are determined by the type of hologram. Generally speaking, there exist two holographic recording geometries, transmission and reflection. Transmission is so called because the reading beam passes through the material to be diffracted. Reflection is so called because the diffracted beam is diffracted back to the same side of the media as the incident reading beam.
As shown in FIG. 4, holography involves writing and reading an interference pattern 210 into a holographic recording media 200. There basically exist two holographic geometries: reflection and transmission characterized by the orientation of the reference beam 220 and the object beam 230 compared to the media 200. From the recording geometry depends the reading geometry where a reading beam 240 is used to display the information as a diffracted beam 250. There could be some part of the reading beam that is not diffracted by the hologram 260. In the transmission geometry, the reading beam 240 goes through the holographic recording media 200 and the diffracted beam 250 emerges on the other side of the media. In the reflection geometry, the reading beam 240 is diffracted back by the interference pattern 210 so the diffracted beam 250 is in the same side of the media 200 than the reading beam 240.
Integral holography is a technique that combines holography and integral imaging. The writing and reading of the 3D image is performed using the diffraction of light principle, but the image is encoded using the integral imaging principle. The full hologram is recorded hogel by hogel. In this case we are using the term “hogel” as a contraction of holographic pixel. Each hogel is recorded by the interference between a reference beam and an object beam. The object beam is modulated by the hogel data calculated according to the image of the scene taken from different position or different angle.
A holographic (diffractive) auto (no glasses) stereoscopic (3D) system using a grey-scale (single-color) transmission geometry as shown in FIG. 5, a laser 300 emits a coherent beam 310 that is split into a reference beam 320 and an object beam 330 by a beam splitter 340. The object beam is expanded by means of a telescope 350 and the beam's amplitude is structured by a device 360 that can be a transparent image, a mask, a spatial light modulator etc. The object beam is then resized by a telescope 370, directed by one or several mirrors 380, and focused by a lens 390 to the holographic recording material 400. That lens 390 is spherical in the case of full parallax and cylindrical in the case of horizontal parallax only. The reference beam 320 is shaped by optics 410 to match the shape of the object beam at the holographic material location, and directed by a mirror 420 to the holographic recording material. After one hogel has been recorded, the material is moved to the next hogel location by a translation stage 430. Controlling electronics 500 ensures the synchronization between the laser 300, the translation stage 430 and the device 360 that structures the object beam. In the case the device 360 that structures the object beam 330 is electronic; a memory 510 could be used to store the hogel data. During recording, the material is positioned or otherwise shielded so that the powerful write beam is not incident on a viewer's eyes for safety.
When all the hogels have been recorded, the material is processed to develop the hologram (if needed) and moved to the reading position 440, where a reading source 450 emits a light beam 460 that is expanded by a telescope 470 and diffracted by the hologram 440 in a diffracted beam 480 toward the viewer's eyes.
In the case of a refreshable holographic recording material, when the hologram needs to be erased, the holographic recording device 400 is moved to the erasing location, where the erasing process occurs. In the case of erasure by light, like with photorefractive materials, an erasing light source 600 emits a beam 610 that is expanded with a telescope 620 and illuminates the whole hologram area. Some materials need a heating process for the hologram to be erased. Some materials like photo thermo plastic, need electrical charging and heating the material to erase the hologram. Once erased, the recording material is moved back to the recording position.